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Abstract 

The  growing  role  of  Air  Force  Air  Logistics  Centers  (ALC)  in  complex, 
multi-year  developmental  programs  increases  the  need  for  effective  cost  and 
schedule  performance  measurement.  In  order  to  effectively  manage  these 
activities,  timely  and  accurate  cost  and  schedule  information  must  be  collected 
and  compared  with  an  established  baseline.  The  ALCs  need  a  new 
management  technique  that  facilitates  this  type  of  performance  measurement. 

A  proven  approach  to  cost  and  schedule  performance  measurement  in 
weapon  system  acquisitions  is  EV.  This  approach,  required  of  contractors  who 
are  subject  to  the  Cost/Schedule  Control  Systems  Criteria  (C/SCSC),  allows  the 
program  manager  to  obtain  cost  and  schedule  variances  at  one  month  intervals 
during  program  execution.  These  variances  corroborate  known  program 
problems,  or  give  a  first  indication  of  potential  problems  within  a  program. 

This  thesis  describes  what  performance  measurements  are  currently  in 
use  and  how  the  EV  concept  can  be  applied  to  organic  maintenance  programs. 
This  document  also  relates  management  challenges  to  the  implementation  of  an 
EV  measurement  system  at  organic  maintenance  facilities. 
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ORGANIC  MAINTENANCE  PERFORMANCE  MEASUREMENT; 

A  NEW  APPROACH 

I.  Introduction 

The  Department  of  Defense  (DOD)  budget,  in  terms  of  purchasing  power, 
has  diminished  markedly  over  the  past  decade.  Both  the  decline  and  breakup  of 
the  Soviet  Union  and  increased  concern  over  the  Federal  deficit  have  driven 
Congress  to  reduce  defense  spending.  Because  of  this  budget  reduction  trend, 
the  Air  Force  is  purchasing  fewer  weapon  systems.  Consequently,  the  Air  Force 
is  beginning  to  concentrate  more  of  its  resources  on  maintaining  existing  war 
fighting  capability  and  upgrading  current  weapon  systems. 

This  approach  is  leading  to  changes  in  the  allocation  of  DOD  budget 
dollars.  For  example,  Air  Force  organic  maintenance  funding  is  projected  to 
increase  in  the  President's  fiscal  year  (FY)  95  budget  (2:4),  and  is  expected  to 
increase  in  future  years.  In  fact,  the  budget  focus  on  organic  maintenance  is  not 
a  unique  Air  Force  situation.  DOD  depot  maintenance  funding  is  projected  to  rise 
20%  in  FY  95  (2:3).  This  boost  is  intended  to  meet  increased  demand  for  future 
modifications  and  repairs  to  be  performed  through  internal  DOD  organic 
maintenance  and  by  industry.  This  trend  toward  continuous  upgrading  can  be 
expected  to  continue  into  the  future. 

Within  the  Air  Force,  the  organization  responsible  for  designing, 
developing,  testing,  fielding,  upgrading,  and  repairing  of  weapon  systems  is  Air 
Force  Materiel  Command  (AFMC),  headquartered  at  Wright  Patterson  Air  Force 
Base  (WPAFB),  Ohio.  AFMC  was  created  in  1992  by  combining  Air  Force 
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Rgure  1 .  Air  Force  Materiel  Command  Organizational  Structure 


2 


Systems  Command  (AFSC)  and  Air  Force  Logistics  Command  (AFLC),  which 
were  responsible  tor  acquisition  and  logistics  support,  respectively.  As  seen  in 
Rgure  1 ,  AFMC  oversees  acquisition  centers  and  ALCs.  Acquisition  centers, 
such  as  the  Aeronautical  Systems  Center  (ASC),  are  primarily  responsible  for 
designing,  developing,  testing,  and  fielding  of  weapon  systems.  ALCs,  such  as 
Warner  Robins  Air  Logistics  Center  (WR-ALC),  are  primarily  responsible  for 
depot  maintenance,  which  is  repairing,  maintaining,  and  upgrading  fielded 
weapon  systems. 

Depot  maintenance  can  be  divided  in  two  categories.  In  the  first 
category,  organic  maintenance,  the  depots  are  responsible  for  the  management 
and  performance  of  maintenance  activities.  In  the  second  category,  contract 
maintenance,  the  contractor  is  responsible  for  the  management  and 
performance  of  maintenance  activities.  The  focus  of  this  thesis  is  performance 
measurement  within  the  organic  maintenance  environment. 

As  shown  in  Figure  2,  organic  maintenance  includes  three  major  types  of 
efforts.  The  first,  routine  maintenance,  is  the  continuous  support  of  fielded 
systems,  including  hardware  repair  and  replacement,  and  software  maintenance. 
Hardware  repair  and  replacement  are  accomplished  at  the  system  or  component 
level,  with  either  parts  fabricated  in-house  or  purchased.  Software  maintenance 
involves  the  upgrade  of  fielded  software  due  to  changing  operational 
requirements. 

The  second  type  of  organic  maintenance,  reliability  and  maintainability 
upgrades,  is  the  improvement  of  selected  fielded  hardware  and  software 
components  or  systems  to  make  them  more  supportable  and  mission  capable. 

The  third  effort,  labeled  here  as  acquisition  activities,  was  previously 
limited  to  the  acquisition  centers.  "Acquisition  activities"  includes  the 
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Figure  2.  Types  of  Activities  Found  at  Organic  Maintenance  Facilities 

performance  of  upgrades  to  existing  weapon  systems,  and  potentially  the 
acquisition  of  weapon  systems  and  components 

ALCs  performing  acquisition  activities  will  require  the  implementation  of 
innovative  management  concepts.  The  acxtuisition  activities  tend  to  be  complex 
and  of  longer  duration  than  the  other  two  categories  of  organic  maintenance 
defined  above.  Therefore,  existing  management  control  tools  must  be  evaluated 
and  modified  as  necessary  to  meet  the  new  challenges.  In  order  to  manage 
these  activities,  cost  and  schedule  information  must  be  collected  and  compared 
with  an  established  baseline.  Management  techniques  and  systems  facilitating 
this  type  of  performance  measurement  will  be  required. 

Earned  Value 

EV  is  a  proven  approach  to  major  weapon  system  cost  and  schedule 
performance  measurement.  Critical  in  the  EV  concept  is  determining  the  value 
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amount  of  work  performed  at  any  point  in  time,  as  expressed  in  terms  of  the 
budgeted  dollars  for  the  same  work  (15:20'21 ).  In  an  EV  system,  task 
performance  is  measured  at  the  lowest  level  of  a  contractor's  detailed  work  plan 
(14:29).  These  measured  values  of  the  work  performed  are  summed  to  obtain 
the  program-level  EV  (14:29).  By  using  the  EV  approach,  the  manager  is 
provided  three  measurements:  the  budget  for  the  work,  the  value  of  the  work 
performed,  and  the  cost  of  the  work  performed.  The  budget  and  the  cost  of  the 
work  performed  are  compared  with  the  value  of  the  work  performed  to  calculate 
variances.  To  ensure  traceability  and  to  aid  problem  identification,  the  variances 
are  calculated  at  lower  levels,  as  well  as  at  the  program  level. 

EV  is  the  key  concept  to  the  C/SCSC.  Through  C/SCSC,  the  program 
manager  obtains  variances  at  defined  intervals,  during  program  execution 
(10:1 1-B-2  -  3).  These  variances  may  corroborate  known  program  problems,  or 
give  a  first  indication  of  potential  problems  within  a  program.  Cost  variances 
(CV)  indicate  if  a  program  is  spending  more  or  less  than  the  value  of  the  work 
performed  to  date.  Schedule  variances  (SV)  indicate  if  the  work  is  being 
completed  when  planned.  If  variances  are  beyond  a  threshold,  the  contractor 
program  manager  must  investigate  the  cause  of  the  variance  and  take  corrective 
action  as  necessary  (10:1 1-B-1-3). 

Management  Control 

When  integrated  with  technical  performance  management,  EV  facilitates 
effective  managem  jnt  control.  The  EV  cost  and  schedule  performance 
indicators  show  current  and  cumulative  trends,  identify  existing  or  potential 
problems,  and  aid  in  estimating  total  cost  at  program  completion.  Thus,  the  EV 
approach  can  be  used  to  accurately  track  program  progress.  This  contrasts  with 
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traditional  methods  of  tracking  which  focus  on  spend  rates  or  other  indicators 
such  as  manpower  availability. 

Competition 

The  change  in  depot  focus  toward  acquisition  activities  is  not  the  only 
reason  better  cost  and  schedule  performance  data  is  needed.  The  introduction 
by  DOD  of  Depot  Maintenance  Competition  (OMC)  also  has  heightened  the 
awareness  of  the  need  for  better  data.  Begun  on  a  trial  basis  in  1 991 ,  these 
competitions  were  classified  as  either  Public-Private  (government-industry)  or 
Public-Public  (government-government).  The  first  type  was  a  competition 
between  organic  maintenance  and  private  industry,  while  the  latter  was  a 
competition  between  two  or  more  DOO  depots.  In  both  competitions,  organic 
maintenance  bidders  were  required  to  develop  proposals  and  to  identify 
methods  of  tracking  performance  comparable  to  those  methods  expected  of 
contractors. 

Public-Private  competition  stirred  much  controversy.  Private  industry 
exerted  pressure  on  the  US  government  to  allocate  to  the  private  sector  some  of 
the  60%  of  the  non-acquisition  organic  maintenance  work  currently  reserved  by 
law  for  the  depots.  Industry  advocates  have  argued  that  open  competition 
between  public  and  private  entities  unfairly  favors  the  public  sector.  AFMC, 
however,  has  defended  Public-Private  competition,  stating  that  significant 
savings  have  been  realized  through  competition  between  the  Air  Force  and 
private  contractors  (23:A6). 

In  addition  to  saving  money,  the  organic  maintenance  competition 
programs  increased  awareness  of  the  need  for  better  cost  and  schedule 
performance  measurements,  such  as  those  offered  by  EV.  However,  the  DOD  is 
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currently  recommending  to  Congress  the  cancellation  of  Pubiic-Private 
competition  (1 1 :462).  It  supports  the  creation  of  a  OOD-wide  core  workload. 
Core  workload  is  defined  as  the  maintenance  work  that  must  be  performed  by 
organic  maintenance  at  the  Service  depots.  Industry  would  compete  for 
non-core  workload.  Interservicing  procedures  will  be  used  in  lieu  of  Public- 
Public  competition  for  core  workload.  The  DOD  is  currently  defining  what 
maintenance  activities  wiil  comprise  core  workload.  (1 1 :462) 


Research  Questions 

Contractors  implement  the  EV  management  concept  to  meet  the 
requirements  of  the  C/SCSC.  EV  concepts  appear  appropriate  for  organic 
maintenance  programs.  To  this  end,  the  authors’  research  focused  on 
performance  measurements  currentiy  in  use  or  planned  for  implementation 
within  the  organic  maintenance  environment.  Special  emphasis  was  given  to  the 
potentiai  impiementation  of  the  EV  approach  through  interfacing  with  or 
supplementing  existing  performance  management  tracking  systems. 

Specifically,  this  thesis  investigated: 

1 .  The  existing  procedures,  processes,  and  systems  used  to 
acquire  cost  and  schedule  performance  data  within  organic 
maintenance. 

2.  The  cost  and  schedule  performance  measurements  used  for 
organic  maintenance  by  the  Air  Force  at  higher  management 
levels. 

3.  When  existing  performance  measurements  were  provided  to 
management  and  if  these  measurements  were  effective  for 
managing  programs  within  the  organic  maintenance  environment. 

4.  Whether  EV  concepts  could  be  implemented  by  organic 
maintenance  organizations  and  how  could  its  implementation  aid 
effective  program  execution. 
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The  research  documented  by  this  thesis  was  conducted  for  two  reasons: 
the  changing  of  organic  maintenance  workload  toward  acquisition  activities,  and 


the  increased  emphasis  on  performance  data  due  to  Public-Public  and 
Public-Private  competition.  To  focus  the  investigation,  the  researchers 
developed  the  above  research  questions.  To  answer  these  questions,  the 
researchers:  conducted  an  in-depth  literature  review  of  EV  and  C/SCSC 
guidance,  interviewed  key  organic  maintenance  managers  and  functional 
personnel,  and  analyzed  organic  maintenance  management  procedures  and 
systems. 

Due  to  resource  constraints,  the  researchers  did  not  validate  the  findings 
through  on-site  evaluations  of  organic  maintenance  management  systems.  The 
researchers’  investigation  was  limited  to  analyzing  documentation  describing  the 
F-1 1 1  Stores  Management  System  (SMS)  Upgrade  program  at  the  Sacramento 
ALC  (SM-ALC). 

Methodology 

Applying  EV  to  the  organic  maintenance  environment  is  a  new  concept. 

In  fact,  there  is  no  documented  example  of  a  DOD  program  fully  implementing 
EV.  The  researchers  investigated  current  and  proposed  AFMC  guidance  on 
performance  tracking  by  meeting  with  logistics  and  financial  management 
personnel.  The  researchers  also  interviewed  managers  of  current  and  proposed 


organic  maintenance  cost  and  schedule  tracking  systems.  Because  the 
investigation  encompassed  such  a  broad  subject  area,  an  exploratory,  high- 
level,  research  approach  was  considered  appropriate  for  this  thesis. 

Thesis  Structure 

This  thesis  is  divided  into  five  chapters,  the  first  of  which  is  this 
Introduction.  Chapter  Two,  Literature  Review,  is  a  review  of  EV  concepts,  as 
applied  to  industry.  The  C/SCSC  are  discussed  and  an  introduction  to  EV  is 
presented.  Chapter  Three,  Methodology,  describes  the  research  approach,  and 
documents  the  scope  of  the  thesis  effort. 

Chapter  Four,  Analysis,  presents  the  information  collected  through 
interviews  and  associated  organic  maintenance  related  documentation.  This 
chapter  describes  current  cost  and  schedule  performance  measurement 
direction,  discussing  the  F-1 1 1  SMS  program  as  an  example,  reviews  the 
proposed  Depot  Maintenance  Management  information  System  (DMMIS),  and 
identifies  changes  to  organic  maintenance  organizational  structures. 

Chapter  Five,  Conclusions,  answers  the  research  questions,  and 
comments  on  the  generalization  of  the  results.  Relevant  observations 
concerning  the  implementation  of  EV  concepts  and  suggestions  for  future 
research  are  then  provided. 
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II.  Litarature  Review 


Overview 

Performance  measurements  may  be  obtained  by  applying  EV  concepts  to 
organic  maintenance  programs.  EV  trend  and  performance  indicators  could  be 
important  tools  that  would  aid  the  effective  controlling  and  reporting  of  program 
cost  and  schedule.  The  DOD  EV  concept  has  evolved  over  the  past  three 
decades  and  a  considerable  body  of  literature  exists  describing  this 
management  concept.  This  literature  review  discusses  the  history  and  basic 
requirements  of  the  C/SCSC  and  the  concept  of  EV. 

C/SCSC  Background 

The  EV  concept  has  been  a  keystone  of  the  000  major  system 
acquisition  for  some  time  and  is  used  by  numerous  Federal  Civilian  Agencies, 
such  as  Oepartment  of  Energy,  Oepartment  of  Commerce,  and  Federal  Aviation 
Agency,  among  others.  The  000,  the  first  government  agency  to  implement  EV, 
mandated  C/SCSC  on  contracts  on  Oecember  22, 1 967.  The  000  detailed  this 
requirement  in  000  instruction  7000.2,  Performance  Measurement  for  Selected 
AcquisKion  (1 2:25).  This  document  contained  the  35  criteria  which  contractor 
management  systems  were  required  to  meet.  The  C/SCSC  require  the 
contractor  to  "define  the  work  required  to  meet  contract  objectives,  assign  the 
work  to  specifically  identified  organizational  elements,  establish  internal 
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schedules  and  budgets,  and  periodically  compare  cost  and  schedule 
performance  indicators  against  pianned  budgets  and  schedules"  (29:21 1 ).  The 
35  criteria  have  remained  basically  unchanged  since  their  introduction  (4:10). 
DOD  Instruction  7000.2  was  superseded  by  DOD  Instruction  5000.2,  February 
23, 1991 .  These  criteria  are  applied  to  Research,  Deveiopment,  Test  and 
Evaluation  (RDT&E)  contracts  exceeding  $60  million,  and  procurement  contracts 
exceeding  $250  miliion,  in  constant  1990  dollars  (10:11-6-2). 


Acceptance  of  the  Criteria 

When  the  C/SCSC  were  first  published,  they  were  generally  viewed  as  "a 
very  positive  step  toward  helping  soive  management  problems"  (1 7:D.1 1 .2).  In 
fact,  the  consensus  is  that  "C/SCSC  requirements  have  been  overwhelmingly 
acknowledged  by  both  government  and  industry  managers  as  representing  good 
management  principles"  (21  :E.2.1 ).  In  1982,  fifteen  years  after  C/SCSC  was  first 
mandated,  the  DOD  contracted  with  the  management  consulting  firm  of  Arthur  D. 
Little  Company  (ADL)  to  independently  review  C/SCSC.  The  stated  purpose  of 
the  study  was  to: 

Determine  the  degree  of  acceptance  and  use  of  the  C/SCSC  by 
defense  contractors  and  government  program  managers;  identify 
problems  and  issues,  the  resolution  of  which  could  lead  to  improvements 
in  the  C/SCSC  and  contract  performance  measurement  reporting 
requirements:  and  recommend  policy  changes  that  will  lead  to  these 
improvements  and  could  be  implemented  by  the  Assistant  Secretary  of 
Defense  (Comptroller).  (6:1-1) 
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The  findings  contained  in  the  report  were  based  on  surveys  and  interviews  with 
DOO  and  contractor  personnel  (6:1-1 ).  The  report  contains  seven  specific 
findings  (6:1-2  -  3)  and  ten  specific  recommendations  for  improving  the 
implementation  of  C/SCSC  (6:IV-2).  The  first  finding  was  that  there  was 
"general  endorsement  of  the  criteria  concept,  both  from  government  and  private 
industry"  and  that  both  groups  "considered  C/SCSC  to  be  effective  and  to 
outweigh  the  costs  involved"  (6:1-2).  Thus,  one  may  conclude  that  the  C/SCSC 
has  general  acceptance  for  use  on  000  programs. 

The  C/SCS  Criteria 

When  C/SCSC  is  implemented  on  a  contract,  the  contractor  must 
demonstrate  that  the  contractor’s  management  system  complies  with  35  criteria. 
The  criteria  are  grouped  into  five  categories:  Organization,  Planning  and 
Budgeting,  Accounting,  Analysis,  and  Revisions  and  Access  to  Oata  (3:2-3  -  2- 
4).  These  five  categories  are  presented  below. 

Organization.  These  organization  criteria  ensure  that  the  contractor  will 
"define  all  authorized  work  and  related  resources  to  meet  the  requirements  of 
the  contract,  using  the  framework  of  the  CWBS  [Contract  Work  Breakdown 
Structure]"  (3:2-3).  The  contractor  must  also  integrate  the  CWBS  with  the 
contractor's  functional  organization  and  "provide  for  the  integration  of  the 
contractor's  planning,  scheduling,  budgeting,  work  authorization  and  cost 
accumulation  systems  with  each  other,  the  CWBS,  and  the  organization 
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staicture"  (3:2’3).  Furthermore,  the  contractor  must  demonstrate  the  ability  to 
comply  with  all  the  aforementioned  requirements  (22:50). 

Planning  and  Budgeting.  These  criteria  do  not  specify  a  particular 
planning  or  scheduling  technique,  but  mandate  that  the  chosen  approach  have 
certain  capabilities.  The  contractor  must  "schedule  the  authorized  work  in  a 
manner  that  describes  the  sequence  of  work  and  identifies  the  significant  task 
interdependencies  required”  (3:2-3).  To  fulfill  the  budgeting  requirements,  the 
contractor  must  also  "establish  and  maintain  a  time-phased  budget  baseline  at 
the  cost  account  level  against  which  contract  performance  can  be  measured" 
(3:2-3). 

Accounting.  The  third  category  applies  to  contractor  recording  of  direct 
and  indirect  costs  incurred  which  apply  to  the  contractor  effort  (3:3-13).  “Such 
costs  must  be  directly  summarized  from  the  level  at  which  they  are  applied  to  the 
contract  through  both  the  WBS  and  functional  organization  stmctures  according 
to  procedures  acceptable  to  DCAA  (3:3-13).” 

Analysis.  The  objective  of  these  criteria  is  to  conduct  performance 
measurement  through  the  calculation  of  variances  and  determining  the  causes 
for  deviations  through  variance  analysis.  The  contractor  determines  the 
variances  by  comparing  both  actual  costs  and  the  cost  of  the  work  scheduled 
with  the  budgeted  cost  of  the  work  performed.  The  Variance  at  Completion 
(VAC)  is  determined  by  comparing  the  Budget  at  Completion  (BAC)  with  the 
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Estimate  at  Completion  (EAC).  BAG,  EAC  and  other  EV  terms  are  defined  in 
Appendix  A. 

Analysis  criteria  require  the  calculation  of  various  quantities  used  for 
comparison  at  the  cost  account  and  higher  level.  Variance  Analysis  requires  the 
availability  of  five  quantities:  Budgeted  Cost  for  Work  Scheduled  (BCWS), 

Actual  Cost  of  Work  Performed  (ACWP),  Budgeted  Cost  for  Work  Performed 
(BCWP),  Budget  at  Completion  (BAC),  and  Estimate  at  Completion  (EAC). 

Revisions  and  Access  of  Data.  The  fifth  category  of  the  criteria  “pertains 
to  revisions  to  planning  which  are  necessitated  either  by  contractual  change  or 
by  internal  conditions  which  require  replanning  within  the  scope  of  the  contract, 
it  also  deals  with  maintaining  the  validity  of  the  performance  measurement 
baseline"  (3:2-4). 

Variance  Calculations.  To  conduct  the  required  analysis,  three  primary 
indicators  must  be  computed.  These  indicator,  BCWS,  BCWP,  and  ACWP  are 
calculated  at  the  cost  account  level,  which  is  the  lowest  management  control 
point  within  the  contractor's  organizational  structure.  Higher  level  comparisons 
determine  summary  contractor  performance  for  each  portion  of  the  contract.  At 
the  cost  account  level,  the  contractor  subtracts  BCWS  from  BCWP  to  obtain  the 
Schedule  Variance  (SV),  and  subtracts  ACWP  from  BCWP  to  obtain  the  Cost 
Variance  (CV)  (3:2.4). 

SV  indicates  work  completion  status,  behind  or  ahead  of  the  planned 
schedule.  However,  one  can  not  assume  that  if  work  is  behind  schedule,  the 
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program  is  behind  schedule.  The  work  may  have  been  scheduled  earlier  than 
required,  or  may  not  be  critical  to  the  success  of  the  program.  The  EV  measure 
of  SV  provides  a  good  method  for  measuring  to-date  schedule  attainment,  but  it 
does  not  measure  the  relative  criticality  of  uncompleted  tasks  (7:60).  Additional 
analysis  must  ascertain  the  true  impact  of  the  schedule  variance. 

CV  shows  whether  the  cost  of  the  work  is  being  performed  at  the  planned 
budget.  The  CV  for  the  entire  contract  Is  the  summation  of  the  CVs  of  the  cost 
accounts.  The  variance  is  a  relative  measure  of  whether  contract  performance 
can  be  obtained  within  the  budgeted  cost  envelope.  The  method  chosen  for 
estimating  the  BCWP,  or  EV,  is  critical  to  the  correct  calculation  of  CV  and  SV. 

Calculation  of  CV  and  SV  permits  the  contractor  to  assess  cost  and 
schedule  performance  separately.  The  traditional  method  of  reporting  cost 
variances  [in  private  industry]  has  been  a  comparison  of  budget  with  the  actual 
expenditure.  It  never  takes  into  account  the  value  or  worth  of  the  accomplished 
work"  (20:A.1 .4).  Thus,  the  traditional  approach  would  show  a  favorable  CV  as 
long  as  the  program  was  spending  its  budget  at  the  anticipated  rate.  The  EV 
approach,  however,  would  give  favorable  CVs  only  if  the  actual  cost  of  the  work 
performed  is  lower  than  the  BCWP.  The  EV  CV,  therefore,  provides  the 
manager  with  a  much  improved  measurement  of  cost  performance. 
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BCWP  represents  EV,  and  is  measured  at  the  cost  account  level  for  direct 


costs.  The  DOD  defines  a  cost  account  as  “a  management  control  point  at 
which  actual  costs  may  be  accumulated  and  compared  to  the  budgeted  cost  of 
the  work  performed.  [A  cost  account  is]  a  control,  because  it  represents  the 
work  assigned  to  one  responsible  organizational  element  on  one  contract  work 
breakdown  structure  element”  (10:1 1-B-2-2).  There  are  three  types  of  cost 
accounts,  as  described  below:  Level  of  Effort  (LOE),  Apportioned,  and 
Measured  (12:119). 

Level  of  Effort.  The  DOD  defines  a  level  of  effort  as  an  “effort  of  a 
general  or  supportive  nature  that  does  not  produce  definite  end  products” 

(10:1 1 -8-2-2).  Fleming  indicates  that  "LOE  activities  are  those  which  are 
necessary  to  a  program,  but  which  are  more  time  oriented  than  task-related" 
(12:120).  Program  management,  security,  and  contract  administration  are 
frequently  considered  as  LOE  activities  .  in  this  type  of  cost  account,  the  BCWP 
always  matches  the  BCWS,  thus  creating  a  zero  SV.  A  CV,  however,  could 
arise  if  the  cost  actuals  (ACWP)  are  not  equal  to  the  budgeted  costs  (BCWP). 

Apportioned  Effort.  Apportioned  effort  is  effort  that  is  not  readily  divisible 
below  the  cost  account  level,  but  is  related  proportionately  to  measured  effort 
(10:1 1  -B-2-1 ).  One  example  of  an  apportioned  effort  would  be  factory 
inspection,  which  is  budgeted  as  a  fixed  percentage,  such  as  eight  percent,  of 
factory  labor.  The  factory  labor,  in  this  example,  would  be  the  reference  base  for 


factory  inspection.  The  assumption  is  that  the  requirement  for  the  apportioned 
effort  is  directly  proportional  to  the  reference  base.  The  EV,  or  BCWP,  will  be 
set  at  the  fixed  percentage  of  the  BCWP  of  the  reference  babe.  In  the  example, 
the  factory  inspection  will  be  credited  eight  dollars  for  each  100  dollars  of  BCWP 
earned  by  the  factory  labor.  Thus,  the  SV  of  the  apportioned  effort  always 
reflects  the  status  of  the  effort's  reference  base.  The  CV  is  the  difference 
between  the  ACWP  for  the  apportioned  effort  and  the  derived  BCWP.  Thus,  an 
apportioned  effort  may  have  a  negative  CV,  even  when  the  reference  base  has  a 
positive  CV. 

Measured.  The  intent  of  the  C/SCSC  guidance  is  to  minimize  the  use  of 
LOE  and  apportioned  effort  cost  accounts.  Measured  effort  consists  of  discrete 
tasks  which  have  a  specific  end  product  or  end  result.  The  majority  of  the  work 
is  expected  to  be  accomplished  in  measured  cost  accounts.  Therefore,  most  of 
the  focus  of  EV  discussions  relate  to  this  type  of  cost  account.  The  EV  for 
unstarted  work  within  a  measured  cost  account  is  zero.  The  EV  for  completed 
work  in  a  measured  cost  account  is  equal  to  the  budgeted  cost  of  that  work.  The 
true  challenge  is  estimating  the  EV  for  work  started,  or  "opened,"  but  not 
completed  ^jrork.  The  POD  Joint  Implementation  Guide  states:  "The  major 
difficulty  encountered  in  the  determination  of  BCWP  is  the  evaluation  of  in- 
process  work"  (3:2-4).  EV  is  a  measure  of  the  work  that  has  actually  been 
accomplished. 
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Earned  Value  Calculation  Methods 

The  criteria  do  not  mandate  specific  methods  for  calculating  the  EV,  but 
do  require  that  the  same  method  used  to  measure  the  BCWS  is  used  to  develop 
the  BCWP.  This  consistency  allows  for  meaningful  comparisons  of  the  budget 
with  the  actual  results. 

There  are  a  number  of  methods  for  determining  BCWP,  the  following  six 
methods  have  gained  general  acceptance;  50/50, 0/100,  Milestone,  Percent 
Complete,  Equivalent  Units  and  Earned  Standards.  Of  these  methods,  the 
Milestone  and  Percent  Complete  are  the  two  most  utilized  by  contractors  with 
approved  C/SCSC  systems.  These  methods  are  defined  in  Appendix  B. 

(12:125) 


Imohcations  for  Air  Force  Organic  Maintenance 

The  general  opinion  of  the  literature  reviewed  was  that  the  EV  approach 
provides  useful  cost  and  schedule  information  to  management.  Additionally,  the 
EV  is  used  commercially  on  many  projects,  large  and  small  (1  ;J.2.5)  (15:23). 

Within  OOD,  the  potential  exists  for  application  of  the  EV  concept.  For 
example,  the  Sacramento  Army  Depot  (SAAD)  planned  to  implement  the  EV 
concept  on  the  Firefinder  Radar  Project  (4:56).  However,  due  to  the  closing  of 
SAAD,  the  program  transferred  to  Tobyhanna  Army  Depot  (TOAD),  and  the  EV 
measurement  attempt  was  abandoned.  The  Army  did  learn,  however,  that 
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proper  implementation  would  require  strong  headquarters  leadership,  new  and 
modified  software  tools,  and  trained  personnel  (4:66-67).  The  Navy  is  also 
investigating  the  implementation  of  EV  concept  in  naval  organic  maintenance 
environments. 

By  applying  the  EV  concept,  the  Air  Force  organic  maintenance 
organizations  can  apply  effective  management  control  to  acquisition  activities 
and  selected  routine  maintenance  programs. 
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The  concept  of  applying  performance  measurement  techniques  used  by 
DOO  on  major  systems  acquisition  to  organic  maintenance  environments  was 
approached  using  exploratory  research  techniques.  This  research  style  is  most 
appropriate  because  little  is  known  of  the  applicability  of  the  EV  approach  to  Air 
Force  organic  maintenance  programs.  As  previously  stated,  there  is  no  known 
example  of  an  organic  maintenance  program  fully  implementing  C/SCSC.  There 
are,  however,  a  growing  number  of  potential  efforts  requiring  cost  and  schedule 
tracking.  Data  very  similar  to  that  collected  under  the  EV  concept  will  be  needed 
to  aid  in  the  management  of  these  programs. 

Approach 

The  C/SCSC  literature  review  identified  research  information  sources 
including  government  reports,  instructions,  directives,  textbooks,  periodicals, 
and  a  thesis.  From  these  materials,  the  researchers  compiled  a  high-level 
summary  describing  the  EV  concept  and  its  implementation.  The  Literature 
Review  describes  how  EV  aides  management  control  of  projects.  From  this 
review,  the  researchers  determined  that  the  EV  concept  could  aid  organic 
maintenance  management  control. 

To  discover  which  performance  measurements  were  currently  used  and 
which  were  planned,  the  researchers  interviewed  key  personnel  at  HQ  Air  Force 
Material  Command  (AFMC)  and  reviewed  current  HQ  AFMC  guidance.  The 


information  collection  focus  centered  on  current  and  future  performance 
measurement  methods  and  systems. 

The  open  diaiogue  approach  yielded  information  that  would  be  otherwise 
obscured.  The  interviewed  personnel  provided  the  high  level  policy  perspective 
on  cost  and  schedule  performance  measurements,  as  well  as  copies  of  current 
and  proposed  guidance  and  direction  to  be  implemented  on  ALC  programs. 

Additional  information  was  obtained  from  an  academic  analysis  of  the  cost 
and  work  measurement  aspects  of  the  F-1 1 1  SMS  Upgrade  program  (5:1 ). 
Although  this  upgrade  was  not  completed,  SM-ALC’s  attempt  to  utilize  the  EV 
concept  on  this  effort  is  a  good  example  of  Air  Force  EV  implementation. 

This  thesis  evaluates  the  F-1 1 1  SMS  Upgrade  program  as  a  case  study 
and  compares  the  program’s  proposed  performance  measurement  system  with 
the  requirements  for  an  EV  measurement  system.  The  case  study  yielded 
results  that  may  be  generalized  and  applied  to  other  organic  maintenance 
programs  attempting  to  implement  EV  measurement. 

To  answer  the  thesis  questions,  the  researchers  collected  information  on 
the  current  ALC  method  for  collecting  cost  and  schedule  performance 
measurements,  a  future  performance  measurement  system  planned  for  organic 
maintenance  environments,  and  the  organizational  structure  supporting  the 
collection  of  useful,  timely,  and  valid  cost  and  schedule  performance 
measurement. 
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IV.  Analysis 


Overview 

EV  can  be  an  important  tool  for  the  ALCs  to  monitor  and  control  program 
cost  and  schedule.  This  chapter  discusses  three  major  topics  related  to  the  EV 
concept.  Rrst,  a  discussion  of  the  ALCs’  current  method  for  collecting  cost  and 
schedule  performance  measurements  will  be  presented.  The  second  topic  is  a 
description  of  OMMIS,  which  is  a  planned  organic  maintenance  performance 
measurement  system  supported  by  AFMC.  The  third  topic  discusses  the 
organizational  structure  supporting  the  collection  of  data.  The  concept  of 
establishing  a  Project  Administration  Officer  (PAO)  position  at  each  of  the  ALCs 
represents  an  example  of  potential  organization  changes. 

Current  Cost  and  Schedule  Performance  Measurement 

Current  documents,  reports  and  systems  are  described  in  the  following 
four  sections.  To  discuss  the  current  measurements  that  are  being  produced  on 
organic  maintenance  programs,  one  must  first  review  the  basic  tasking 
documents  and  cost  data  collection  processes.  Therefore,  the  first  section 
describes  the  Work  Assignment  Document  (WAD),  and  basic  cost  data 
collection  processes.  The  second  section  discusses  a  currently  used  cost  and 
schedule  tracking  report,  the  Depot  Maintenance  Performance  Tracking  Report 
(DMPTR).  The  third  section  describes  the  Programmed  Depot  Maintenance 
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Scheduling  System  (PDMSS),  a  scheduling  system  in  use  at  Air  Force  organic 
maintenance  organizations,  (28).  The  fourth  section  details  an  attempt  to 
perform  EV  measurement  on  the  F-1 11  SMS  Upgrade  Program  at  SM-ALC. 

Workload  Assignment  Document.  The  Workload  Assignment  Document 
(WAD)  is  the  basic  Air  Force  agreement  document  between  the  buyer 
(customer)  and  the  seller.  The  buyer  is  often  a  major  using  command,  and  the 
seller  is  typically  the  ALC.  The  WAD,  ranging  from  several  to  several  dozen 
pages,  defines  the  terms  and  conditions,  such  as  the  fixed  cost  and  schedule, 
for  the  work  to  be  done  (9).  The  WAD  terms  are  determined  through 
negotiations  between  the  buyer  and  the  seller. 

An  example  of  a  WAD  would  be  an  Air  Combat  Command  (ACC)  request 
for  Ogden  ALC  (OO-ALC),  to  upgrade  the  F-1 6  landing  gear  at  a  negotiated  cost 
and  schedule.  Another  example  of  a  WAD  could  be  a  flight  computer  upgrade. 
The  F-1 1 1  SMS  Upgrade  Program  was  a  weapon  system  upgrade  acquisition 
activity  executed  through  a  WAD.  An  example  of  a  WAD  executed  on  this 
program  is  provided  in  Appendix  C. 

After  the  customer  and  the  depot  agree  to  the  scope  of  effort  to  be 
accomplished  within  a  period  of  time  for  a  specific  cost,  the  customer  funds  that 
work  through  a  Project  Order  (PO).  The  PO  provides  the  funding  required  and 
identifies  the  units  affected.  When  the  PO  is  received,  the  budget  and  schedule 
Information  is  loaded  Into  the  G004L  Job  Order  Production  Master  System.  This 
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system  is  one  of  over  twenty  “legacy”  systems,  currently  operating  at  Air  Force 
oi^anic  maintenance  facilities,  used  to  collect  and  process  cost  and  budget  data. 

When  the  PO  provides  the  funding,  work  begins,  and  costs  are  collected 
as  the  units  are  produced.  The  primary  computer  system  responsible  for  the 
collecting  actual  costs  is  the  G072A  Depot  Maintenance  Production  Cost 
System.  The  overhead  rates  are  included  in  the  costs,  and  the  customer  is  then 
billed  through  the  G004B  Project  Order  Control  System.  Both  the  G072A  and 
the  G004B  are  legacy  computer  systems.  The  customer  is  billed  at  the  agreed 
upon  WAD  price  as  units  are  completed.  (9) 

Standard  billing  rates  are  used  in  the  WAD.  This  is  because  DOD  depots 
use  a  two  year  stabilized  rate  system  to  support  the  customer’s  funding  request 
cycle.  This  funding  system  requires  the  depot  to  submit  one-year  rate  quotes  for 
future  years.  The  rate  quote  is  provided  up  to  two  years  before  the  work  begins. 

Depot  Maintenance  Performance  Tracking  Report.  The  procedures 
described  above  result  in  cost  data  collection  and  actual  cost  realization 
primarily  after  a  unit  is  produced.  Therefore,  the  actual  cost  data  are  not 
compared  with  the  budgeted  cost  data  in  any  systematic  process  during  program 
execution. 

To  facilitate  collecting  cost  and  schedule  performance  information,  AFMC  has 
developed  the  DMPTR.  This  report,  shown  in  Rgures  3  and  4,  consists  of  a 
schedule  and  cost  report  and  requires  an  assimilation  of  existing  legacy  system 
information.  The  DMPTR  provides  cost  and  schedule  information  primarily  from 
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existing  management  systems.  This  thesis  discusses  data  contained  in  the 
OMPTR.  Although  terms  such  as  "earned  value"  are  absent  from  the  OMPTR,  it 
contains  data  similar  to  that  found  in  an  EV  system. 

The  OMPTR  approach  consists  of  two  reports,  one  for  schedule  and  one 
for  cost.  CVs  and  SVs  are  cumulative.  The  convention  of  a  negative  variance 
meaning  behind  schedule  or  over  cost  is  the  same  for  EV  techniques.  Figures  3 
and  4  data  identify  work  behind  schedule  and  over  budget. 

To  fully  explain  these  two  reports,  the  thesis  describes  some  of  the 
unique  terms  used  in  the  report.  The  follovwng  discussion  of  these  two  reports  is 
based  on  preliminary  HQ  AFMC  guidance  (9).  The  heading  and  ending 
signature  portions  of  the  reports  contain  seif  explanatory  administrative  data,  but 
the  terms  used  in  the  bodies  need  to  be  described.  The  schedule  and  cost 
reports  contain  columns  representing  the  fiscal  year  months. 

Schedule  Report.  The  schedule  report  provides  summary  schedule 
information.  The  first  line  in  the  spreadsheet  section  of  the  schedule  report  is  for 
the  Net  WAD  Quantity  (NWQ).  The  NWQ  is  the  cumulative  number  of  items 
required  by  the  WAD,  spread  out  on  a  monthly  basis.  The  next  line,  "CUM  QTY 
INDUCT"  represents  the  cumulative  quantity  of  items  that  have  been  inducted, 
or  started,  into  the  depot  repair  system.  This  figure  is  available  in  the  G004L  or 
the  G1A  reports  currently  in  use.  "CUM  QTY  SCHED"  requires  a  manual 
calculation  and  represents  the  number  of  items  that  have  been  scheduled  for 
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(SAMPLE  FORMAT) 
DMC  SCHEDULE  REPORT 


ALC:  REPORT  AS  OF  DATE 

PRODUCT  DIRECTORATE:  WAD  NUMBER: 

WORKLOAD  TITLE:  PERIOD  OF  PERFORMANCE: 

UNIT  SHOP  FLOW  DAYS: 30 
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TITLE: 


SIGNATURE: 


DATE: 


Figure  3.  Defense  Maintenance  Competition  Schedule  Report 


26 


(SAMPLE  FORMAT) 
DMC  COST  REPORT 


ALC:  REPORT  AS  OF  DATE: 

PRODUCT  DIRECTORATE:  WAD  NUMBER: 

WORKLOAD  TITLE:  PERIOD  OF  PERFORMANCE: 


N 

0 

J 

F 

m 

OH 

m 

J 

J 

A 

S 

AT  COMPL 

NWV 

m 

7000 

11500 

ireoo 

27500 

31000 

35000 

CUM  SALES 

■i 

1200 

2500 

7400 

9800 

CUM  COST 

0 

1355 

2625 

6175 

7705 

9950 

PROF/(LOSS) 

m 

-155 

mg 

-175 

-305 

-150 

0 

0 

0 

wm 

COST  VAR  % 

-13% 

-5% 

-3% 

-4% 

-2% 

TITLE: 


SIGNATURE: 


DATE: 


Rgure  4.  Defense  Maintenance  Competition  Cost  Report 


* 


27 


completion  during  each  month.  The  CUM  QTY  SCHEO  is  analogous  to  BCWS. 
(9) 

The  next  line,  "CUM  QTY  PROD,"  cumulative  quantity  produced,  is  the 
number  of  items  actually  produced  during  that  month.  This  quantity  is  the  total 
BCWP  for  completed  items  for  the  month  expressed  in  terms  of  item  count. 

The  fifth  line,  "CUM  SCHED  VAR,"  cumulative  SV,  and  represents  the 
difference  between  the  number  of  items  produced  and  the  number  of  items 
scheduled. 

The  sixth  and  final  line  on  the  schedule  report  is  the  "SCHED  VAR  %," 
which  is  the  cumulative  SV  percentage.  This  is  obtained  by  dividing  the  CUM 
SCHED  VAR  by  the  CUM  QTY  SCHED,  and  is  graphed  directly  above  the 
spreadsheet  area. 

Cost  Report.  The  cost  report  requires  the  calculation  of  quantities 
analogous  to  EV  quantities.  As  in  the  schedule  report,  the  top  and  bottom  of  the 
report  are  self  explanatory,  and  the  spreadsheet  quantities  require  explanation. 
The  first  line,  NWV,  is  the  Net  Work  Value  that  is  the  actual  dollar  budget  figure 
for  the  WAD  if  completed  as  planned. 

The  next  line,  CUM  SALES,  is  the  cumulative  sales  that  represent  the 
amount  that  will  be  billed  to  the  customer  from  the  Q004B  Project  Order  Control 
System.  The  seller  bills  the  customer  at  the  agreed  to  sales  price  for  items 
completed. 
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The  third  line,  CUM  COST,  is  obtained  from  the  G072A  Depot 
Maintenance  Production  Cost  System  and  represents  the  cumulative  costs 
through  that  month  on  this  WAD.  CUM  COST  is  directly  comparable  with 
ACWP.  The  fourth  line,  PROFIT/{LOSS),  shows  whether  the  price  charged  to 
the  customers  was  above  or  below  the  costs  incurred.  PROFIT/(LOSS)  is 
calculated  in  the  same  manner  as  the  EV  CV  is  computed  and  impacts  future 
pricing  rates.  The  last  line,  COST  VAR  %,  is  the  cumulative  CV  percentage, 
which  is  calculated  by  dividing  the  Profit/Loss  by  cumulative  sales.  This  quantity 
is  graphed  directly  above  the  spreadsheet  portion. 

The  DMPTR  is  not  a  true  performance  report.  It  does,  however,  represent 
an  attempt  toward  an  interim  management  tool,  approaching  an  EV  concept 
implementation.  As  noted,  the  DMPTR  does  not  include  the  progress  on 
partially  completed  units,  or  work  in  process,  and  therefore  does  not  equate  to 
EV  performance  measurement.  To  obtain  more  complete  information,  the  ALCs 
will  need  to  implement  a  cost  and  schedule  control  system  that  can  collect  and 
incorporate  data  on  work  in  process.  The  DMMIS  system,  discussed  later,  when 
fully  implemented  may  provide  work  in  process  data. 

Programmed  Depot  Maintenance  Scheduling  System.  Although  many 
program  specific  management  tools  and  computer  programs  exist  to  help 
managers  track  schedules,  the  Programmed  Depot  Maintenance  Scheduling 
System  (PDMSS)  is  one  that  is  widely  used  within  Air  Force  depots  (5:4).  The 
PDMSS  has  the  capability  to  perform  resource  loading,  resource  leveling,  and 
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‘Nvhat  if  analysis  when  manpower/materiai  requirements  and  availability  are 
input  (5:4). 

“The  [POMSS  scheduling]  network  is  based  on  the  activities  generated  by 
the  depot’s  G037  system.  This  system  generates  standard  hours  for  each  of  the 
tasks  required  to  maintain  the  airframe.”  Precedence  relationships  are  then 
taken  into  consideration  to  develop  “the  earliest  and  latest  start  and  finish  dates 
for  each  task,  as  well  as  the  critical  path  (5:4).”  Actual  and  standard  hours 
completed  are  recorded.  These  standard  hours  completed  are  the  BCWP  for 
the  job.  (5:4) 

POMSS  does  not  provide  on  going  schedule  measurements,  except  for 
standard  hours.  The  measurement  of  ongoing  schedule  status  requires  a  more 
comprehensive  performance  measurement  system.  While  POMSS  does 
represent  an  available  scheduling  system,  it  does  not  utilize  EV  relationships 
and  can  be  considered  an  interim  tool  pending  application  of  the  EV  concept  at 
the  ALCs. 

F-1 1 1  Stores  Management  System  Upgrade  Program.  The  F-1 1 1  SMS 
Upgrade  Program  represented  an  attempt  to  replace  system  components 
utilizing  current  technology  and  using  in-house  SM-ALC  resources.  The 
Program  was  established  under  system  acquisition  procedures  and  included 
both  development  and  production  phases,  the  cost  of  which  was  estimated  at 
approximately  $100  million.  Because  the  remaining  F-1 1 1  aircraft  in  the  Air 
Force  inventory  were  designated  for  retirement,  the  program  was  canceled. 
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There  were  numerous  organizational  and  system  challenges  present  in 
assuming  responsibility  for  this  acquisition  activity  effort.  Aside  from  the 
technical  tasks,  the  need  for  management  cost  and  schedule  data  was  evident. 

On  the  F-1 1 1  SMS  Upgrade  program,  the  primary  software  tool  used  in 
the  compilation  of  EV  quantities  was  a  common  spreadsheet  program.  The 
financial  manager  collected  BOWS,  BCWP  and  ACWP  figures  from  the  software 
development  and  engineering  organizations  working  on  the  program.  The 
financial  data  was  then  entered  into  the  spreadsheet.  (5:1) 

The  F-1 1 1  SMS  Upgrade  example  illustrated  that  to  implement  a 
successful  EV  approach  to  cost  and  schedule  performance  measurement,  one 
should  not  only  collect  data,  but  also  ensure  that  the  data  collected  is  valid.  The 
financial  manager  collected  data  as  required,  but  was  unable  to  assure  the 
data’s  validity.  To  accomplish  more  accurate  data  collection,  program  personnel 
must  be  trained,  adequate  computer  resources  must  be  acquired,  and  program 
structure  must  incorporate  work  small  enough  to  accurately  assign  EV. 

Depot  Maintenance  Management  Information  System 

As  illustrated  by  the  F-1 1 1  SMS  Upgrade  program,  implementing  an  EV 
program  without  the  proper  software  tools  can  be  frustrating  and  difficult.  A 
system  that  will  aid  the  implementation  of  EV  on  future  routine  maintenance  and 
acquisition  activity  programs,  is  the  DMMIS.  This  system  will  eventually  replace 
the  many  legacy  systems. 
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The  Air  Force  concept  for  DMMIS  operation  is  to  provide  production 
information  services  for  depots  similar  to  those  production  information  services 
provided  by  Manufacturing  Reso-urces  Planning  (MRP  II)  systems  used 
commercially.  DMMIS  uses  a  common  data  base  to  interactively  tie  six 
functional  areas:  general  ledger,  time  and  attendance,  budget,  customer  order 
management,  and  cost  management.  (16). 

The  following  DMMIS  discussion  is  divided  into  two  parts.  The  first  part 
covers  DMMIS  programmatic  issues.  Programmatic  issues  include  the  schedule 
status  of  the  DMMIS  program,  the  possibility  for  DMMIS  configuration  changes, 
the  possibility  of  DMMIS  deployment  across  Air  Force  wide  depots  or  DOD  wide 
depots,  and  identification  of  DMMIS  program  management  responsibility.  The 
second  part  discusses  DMMIS  technical  issues,  including  currently  planned 
DMMIS  capabilities  and  DMMIS  flexibility  for  possible  future  capabilities  to 
improve  the  manageability  of  depot  programs.  The  technical  issues  are 
presented  with  a  perspective  on  the  comparison  between  DMMIS  capability  and 
the  EV  performance  measurement  concept. 

Programmatic  Information.  In  1992,  the  DOD  selected  DMMIS  as  the 
standard  cost  performance  measurement  system  for  military  depot  maintenance 
operations  for  commodities.  The  Joint  Logistics  Systems  Center  (JLSC), 
WPAFB,  Ohio,  a  DOD  organization,  assumed  management  of  DMMIS  from  the 
Air  Force  on  July  1 , 1994. 
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DMMiS  was  installed  on  a  test  trial  basis  at  OO-ALC,  Hill  AFB,  Utah,  in 
late  1993,  to  measure  cost  performance  of  the  C-5A  landing  gear  maintenance 
program.  During  the  first  phase  of  DMMIS  operation  on  the  program,  OO-ALC 
operated  the  old  legacy  system  in  parallel  with  DMMIS  to  ensure  that  the  DMMIS 
results  were  accurate.  DMMIS  now  operates  at  OO-ALC  at  the  C-5A  landing 
gear  maintenance  depot,  on  a  standalone  basis. 

Technical  Information.  DMMIS  is  being  developed  to  meet  the  unique 
conditions  encountered  in  depot  environments.  These  include  the  method  for 
managing  floating  inventory  and  providing  material  management  cost  data.  (16) 

DMMIS  also  supports  all  CV  measurements.  Although  the  current  DMMIS 
configuration  provides  schedule  status  information,  it  does  not  compare 
schedule  status  information  with  the  planned  (or  budgeted)  program  schedule. 
(16) 

DMMIS  Capabilities.  DMMIS  capabilities  provide  flexibility  to  track 
anticipated  complex  weapon  system  acquisitions  and  upgrades.  A  critical  key  to 
the  ability  of  EV  to  accurately  measure  performance  on  work  in  process  is  the 
organization  of  the  overall  task  into  cost  accounts.  The  effectiveness  of  EV  as  a 
management  tool  is  accomplished  through  the  establishment  of  responsible 
managers  for  the  established  cost  accounts.  DMMiS  demonstrates  similar 
characteristics  and  capability. 

Bill  of  Material.  DMMIS  organizes  work  orders  by  the  bill  of  material 
(BOM).  In  routine  maintenance  activities,  DMMIS  maintains  the  parent  to  child 
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relationships  between  parts,  subassemblies,  and  assemblies,  to  three  levels, 
while  fulfilling  the  floating  inventory  requirements  of  efficient  depot  operations. 
(16) 

Workcenter  File.  DMMiS  retains  organizational  relationships  to  seven 
levels,  the  lowest  of  which  is  the  workcenter.  The  levels  of  organization  from 
lowest  to  highest  are:  workcenter,  resource  control  center  (RCC),  section, 
branch,  division,  directorate,  and  center.  OMMIS  manages  a  maximum  of  38 
attributes  for  each  workcenter,  four  of  which  are  currently  mandatory: 
workcenter  identification,  workcenter  description,  workcenter  type,  and  RCC  link. 
Other  currently  used  attributes  include  primary  resource,  secondary  resource, 
personal  fatigue  and  delay  factor  (PF&O),  capacity,  queue  time,  and  move  time 
(16). 

Employee  Master  File.  DMMIS  maintains  personnel  information  that 
supports  a  clear  structure  of  authority  and  accountability.  DMMiS  maintains  a 
maximum  of  25  employee  attributes  for  each  employee  in  the  operation.  Ten 
attributes  for  each  employee  are  currently  mandatory:  employee  identification, 
which  is  linked  to  the  social  security  number;  employee  labor  rate;  employee 
production  acceptance  certification  (PAC)  codes,  indicating  the  employee’s 
qualifications;  the  work  center  to  which  the  employee  is  assigned;  the  foreman 
assigned  to  the  employee;  assigned  skill  <x}des  (1 6). 

Cost  Accumulation.  DMMiS  accumulates  costs  at  the  workcenter  level 
from  four  areas,  including  labor,  material,  production  overhead,  and  general  and 
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administrative  (G&A)  overhead.  OMMIS  cost  collection  supports  actual  costs, 
planned  costs  for  the  establishment  of  standards  (budgeted  costs),  and  frozen 
costs  for  the  establishment  of  two  year  stabilized  rates.  The  CV  is  calculated  as 
the  standard  cost  ,or  BCWP,  minus  the  actual  cost,  or  ACWP.  CVs  at  the  work 
center  level  are  rolled  up  to  the  project  level  to  indicate  overall  project 
performance. 

While  DMMIS  could  be  modified  to  combine  baseline  and  current  cost 
information  to  calculate  SV,  it  currently  only  provides  detailed,  real  time  cost 
performance  measurement.  However,  it  appears  DMMIS  can  be  modified  to 
calculate  SV  information.  (16) 

The  Project  Administration  Office 

While  computer  support  systems  are  critical  to  obtaining  current  cost  and 
schedule  information,  management  control  systems  must  also  be  in  place  that 
can  act  on  this  information.  The  current  ALC  organizational  structures  are  being 
evaluated  to  identify  possible  changes  that  may  facilitate  better  management  of 
maintenance  and  weapon  system  upgrade  programs. 

In  programs  won  through  a  Public/Private  competition,  concerns  were 
raised  about  administration  oversight.  The  PAO  concept  was  developed  in 
response  to  these  concerns,  and  in  order  to  facilitate  better  management 
practices.  The  PAO  was  envisioned  as  a  high  level  administrative  element 
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Air  Logistics 
Center  Commander 


Rgure  5.  Project  Administration  Office  Organizational  Structure 


reporting  directly  to  the  ALC  commander.  A  sample  PAO  organizational 
stmcture  is  shown  in  Figure  5.  (28) 

Stimcn^ry 

Many  changes  are  taking  place  in  the  organic  maintenance  environment. 
ALCs  are  ^ced  with  changing  workloads.  These  involve  acquisition  activities 
and  inter-service  department  competitions  for  major  system  acquisition  and 
routine  maintenance  programs,  in  order  to  aid  programs  fulfilling  new  data 
requirements,  HQ  AFMC  and  the  JLSC  are  promoting  advanced  computer 
systems,  such  as  DMMIS. 

Rnally,  the  reports  and  data  generated  can  help  ALC  program 
management  if  used  in  the  management  control  of  programs.  Of  course,  the 
data  must  also  be  timely  and  valid.  Because  of  this,  new  organizational 
structures  will  be  needed  to  help  ensure  program  status  measurement  is  valid. 
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Overview 


V.  Conclusions 


This  thesis  focused  on  the  application  of  the  EV  concept  at  Air  Force 
ALCs,  and  identified  trends  in  organic  msuntenance  management.  For  the  first 
time,  cost  and  schedule  performance  measurement  similar  to  EV  is  being 
considered  for  application  on  Air  Force  organic  maintenance  programs. 

In  this  era  of  OOD  drawdowns,  the  Services  are  seeking  to  become  even 
more  competitive.  Applying  EV  to  organic  maintenance  programs  can  improve 
the  competitiveness  of  the  implementing  depots. 

The  discussion  of  this  dynamic  management  opportunity  is  organized  into 
three  areas.  The  first  reiterates  the  Research  Questions  and  provides  answers 
to  them.  The  second  area  discusses  findings  resulting  from  the  data  collection. 
The  third  and  final  area  details  opportunities  for  future  research. 

Research  Questions 

The  thesis  sought  to  answer  four  research  questions.  These  questions 
were  answered  through  a  combination  of  a  general  literature  review,  compilation 
of  Air  Force  guidance,  and  personal  interviews.  The  thesis  investigations  are 
listed  below. 

The  existing  procedures,  processes,  and  systems  used  to  acquire  cost 


requirement  for  cost  and  schedule  performance  data  is  envisioned  as  the 
DMPTR.  This  report  can  be  created  on  a  monthly  basis,  and  can  be  applied  on 
programs  competed  with  industry.  This  report  can  be  prepared  on  a 
spreadsheet  program,  using  data  from  the  existing  Air  Force  legacy  computer 
systems. 

Numerous  computer-based  systems  are  involved  in  cost  tracking.  The 
existing  web  of  systems  used  by  Air  Force  organic  maintenance  organizations  is 
complicated  and  provides  insufficient  data  for  EV  measurement.  DMMIS  can 
replace  many  of  the  existing  systems  and  could  support  EV  measurement. 

The  cost  and  schedule  performance  measurements  used  for  organic 
maintenance  bv  the  Air  Force  at  higher  management  levels.  The  current 
measurements  in  use  by  the  ALCs  are  obtained  from  the  several  legacy 
systems.  These  measurements  include  CVs  and  SVs  along  with  cost  and 
schedule  percent  variances.  Many  of  these  quantities  are  related  to  EV 
quantities,  or  can  be  converted.  As  the  DMMIS  system  is  implemented  at  all 
depots,  the  cost  and  schedule  reporting  requirements  can  be  fulfilled  by 
automated  DMMIS  reports.  Organic  maintenance  management  would  then  have 
a  more  effective  management  tool.  DMMIS,  with  some  changes,  could  compute 
all  EV  quantities. 

When  existing  performance  measurements  were  provided  to  management 
ind  if  these  measurements  were  effective  for  manaaino  programs  within  the 
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collected  by  different  systems,  at  different  times,  in  various  formats,  and  are  not 
a  part  of  a  common  management  system,  organic  maintenance  management 
cannot  use  the  data  for  cost  control.  Additionally,  there  is  no  formal 
management  system  capable  of  accurately  determining  schedule  progress. 

In  contrast,  new  cost  and  schedule  measurements  as  identified  in  reports 
such  as  the  DMPTR  should  provide  a  good  basis  for  the  analysis  of  cost  and 
schedule  performance.  The  DMPTR  is  not  a  true  performance  report  because 
some  cost  information  is  not  current  and  credit  is  not  given  for  partially 
completed  units.  The  Findings  section  of  this  chapter  details  these  and  other 
implementation  issues. 

Whether  EV  concents  could  be  implemented  bv  omanic  maintenance 
organizations  and  how  could  its  implementation  aid  effective  program  execution. 
EV  measurement  allows  management  control  and  should  be  implemented  on 
specified  organic  maintenance  programs.  The  implementation  of  DMMIS  could 
simplify  EV  measurement  of  organic  maintenance. 

EV  has  proven  to  be  a  useful  management  tool  on  DOD  acquisition 
programs.  This  is  especially  true  in  the  management  of  high  cost,  multi-year, 
complex  programs.  Air  Force  organic  maintenance  organizations,  which  are 
increasingly  performing  complex  weapon  system  acquisitions  and  upgrades,  will 
benefit  substantially  by  using  the  EV  techniques  in  their  management  approach. 


40 


Findings 

This  section  details  seven  findings  that  relate  to  the  implementation  of  EV 
to  organic  maintenance  management.  A  discussion  of  the  factors  impacting 
implementation  follows. 

A  Competitive  Edge.  Improving  cost  and  schedule  performance 
measurement  provides  a  competitive  edge  that  ALCs  should  implement  and 
maintain.  Along  with  having  modern,  cost  efficient  facilities,  the  improved 
performance  measurement  ability  enhances  the  ALCs’  capabilities  to  attract 
organic  maintenance  work. 

Qraani2ational  Structure.  As  cost  and  schedule  performance  data  are 
collected,  one  must  be  sure  that  the  data  is  valid  and  accurate.  In  order  to 
assure  that  these  data  are  useful,  Air  Force  management  is  reviewing 
organizational  structures  necessary  to  execute  program  direction.  The  PAO 
initiative  is  one  example  of  potential  organizational  structure  changes.  Other 
changes  may  result  from  the  ongoing  examination  of  depot  organizational 
structures  due  to  changing  workloads.  The  examination  is  focusing  on 
organizational  changes  that  facilitate  the  timely  collecting,  reporting,  and 
analyzing  of  performance  measurement  data. 

Training.  There  may  be  a  need  to  provide  tailored  EV  training  to  Air 
Force  personnel  in  the  organic  maintenance  environment.  Because  DODI 
5000.2  requires  C/SCSC  on  high  cost  RDT&E  and  procurement  contracts,  the 
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program  offices  and  supporting  staff  are  familiar  with  the  EV  concept.  Because 
the  ALCs  do  not  apply  C/SCSC,  an  understanding  of  the  EV  concepts  may  be 
low. 

If  the  depots  are  to  actively  use  EV  measurement  as  a  tool,  the  organic 
maintenance  community  must  be  trained.  Various  organizations,  such  as  the  Air 
Force  Institute  of  Technology,  have  existing  courses  to  teach  EV  concepts  and 
could  provide  this  training. 

DMMIS  Implementation.  OMMIS  is  a  powerful  tool  for  the  collection  of 
timely  cost  data.  However,  the  system  does  not  automatically  compute  SV  and 
percent  SV.  The  system  could  easily  be  modified  to  compare  BCWP  with  BCWS 
to  obtain  SVs  (16).  The  JLSC  hopes  to  implement  DMMIS  at  all  DOD  depots. 

EV  Data  Collection.  Many  of  the  cost  data  collection  systems  operate 
with  a  time  lag,  which  does  not  allow  current  costs  to  be  known.  This  lag 
impedes  the  ability  of  a  manager  to  use  cost  data  for  the  control  of  resources, 
because  timely  changes  that  improve  cost  and  schedule  performance  cannot  be 
made.  EV  implementation  without  timely  data  may  be  a  wasted  exercise. 

EV  Guidelines.  Contractors  provide  cost  and  schedule  information  in 
accordance  with  C/SCSC.  Because  concepts  such  as  baseline  control  and 
overhead  measurement  must  be  defined  in  an  organic  maintenance  context, 
implementation  guidelines  or  criteria  should  be  made  available  to  organic 
maintenance  organizations.  EV  performance  measurement  systems  would  be 
developed  and  approved  in  accordance  with  these  guidelines. 
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Scope  of  Depot  Work.  Under  the  concept  of  integrated  Weapon  Systems 
Management  (IWSM),  some  of  the  duplication  of  responsibility  for  weapon 
systems  ended.  Each  major  system  has  a  single  program  manager  with  overall 
program  responsibility.  For  example,  while  the  F-16  A  and  B  models  were 
managed  at  Hill  AFB,  UT,  the  more  current  C  and  D  models  were  managed  at 
Wright-Patterson  AFB,  OH.  Now,  there  is  a  single  program  manager  for  the 
entire  F-16  program  who  resides  at  Wright-Patterson  AFB,  OH.  In  other  cases, 
such  as  the  F-1 5  program,  the  program  manager  resides  at  the  ALC. 

in  the  past,  when  an  ALC-managed  weapon  system  required  major 
modifications  and  updates,  the  acquisition  centers  were  usually  called  on  to 
contract  out  the  work.  However,  an  ALC  program  manager  may  choose  to  have 
the  major  modification  done  in-house. 

The  modification  work  performed  by  ALCs  has  become  increasing 
complex  and  costly.  These  are  the  types  of  programs  that  benefit  most  fully  from 
the  implementation  of  EV.  The  best  opportunity  for  cost  and  schedule 
performance  improvement  is  the  application  of  EV  to  these  large,  multi-year 
efforts,  although  smaller  and  more  simple  programs  should  benefit  as  well. 

Areas  For  Future  Research 

This  thesis  has  dealt  with  many  different  issues  and  topics  related  to  EV 
and  organic  maintenance  operations.  Five  areas  of  interest  warrant  future 
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study,  but  were  outside  the  scope  of  this  thesis.  A  discussion  of  these  areas 
follows. 

Manaaemenfs  Use  of  EV  Data.  The  most  important  suggestion  for  future 
research  focuses  on  how  management  will  use  EV  data  once  the  EV 
performance  measurement  system  has  been  implemented.  Additional  research 
could  document  how  EV  data  is  used  by  management  to  improve  cost  and 
schedule  performance. 

Case  Studies.  The  researchers  were  unable  to  conduct  in  depth  case 
studies  of  actual  programs  trying  to  implement  EV.  An  example  of  Air  Force  EV 
implementation  lO  an  organic  maintenance  program  was  the  F-1 1 1  Storage 
Management  Systems  (SMS)  Upgrade  program.  This  example  identified  some 
difficulties  that  may  be  encountered  when  implementing  EV  in  the  organic 
maintenance  environment.  A  follow-up  study  of  this  or  another  program  would 
add  credibility  to  the  conclusions  and  findings  of  this  thesis. 

EV  Implementation.  Guidelines  explaining  the  implementation  of  EV 
performance  measurement  in  the  organic  maintenance  environment  has  not 
been  developed.  Additional  research  is  required  to  determine  what  guidelines 
would  benefit  the  organic  maintenance  organizations  in  implementing  the  EV 
concepts. 

Current  Systems  and  EV.  While  this  thesis  described  some  of  the  major 
existing  computer  systems  involved  in  cost  data  collection,  a  more  thorough 
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analysis  of  these  systems  would  determine  whether  these  systems  could  be 
modified  to  support  EV  measurement. 


DMMIS  SuDDortino  EV.  DMMIS,  once  implemented,  will  facilitate  EV 
measurement.  However,  the  program  is  not  currently  being  managed  with  the 
implementation  of  EV  as  a  requirement.  Additional  research  could  determine 
exactly  how  much  effort  is  required  to  modify  DMMIS  to  automatically  calculate 
all  EV  quantities  as  defined  by  C/SCSC. 

Summanr 

The  declining  DOD  budget  and  the  changing  role  of  organic  maintenance 
present  new  challenges  to  Air  Force  leadership.  One  promising  approach  to 
meet  these  challenges  is  the  implementation  of  improved  cost  and  schedule 
performance  measurement,  based  upon  the  EV  concept,  on  organic 
maintenance  programs.  The  Air  Force,  as  well  as  other  Services,  has  saved 
money  and  gained  management  control  by  applying  the  EV  performance 
measurement  concept  to  acquisition  programs.  Similar  benefits  can  be  expected 
by  applying  the  EV  concept  to  DOD  organic  maintenance  programs. 
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These  definitions  are  directly  extracted  from  IX)D  Instruction 
5000.2,  pp  11-B-2:1  -  11-B-2-3: 

1 .  Budgeted  Cost  for  Work  Scheduled  (BCWS)  is  The  sum  of  budgets 
for  ail  work  packages,  planning  packages,  etc.,  scheduled  to  be 
accomplished  (including  in-process  work  packages),  plus  the  amount  of 
level-of-effort  and  apportioned  effort  scheduled  to  be  accomplished  within 
a  given  time  period.” 

2.  Actual  Cost  of  Work  Performed  (ACWP)  is  The  cost  incurred  and 
recorded  in  accomplishing  the  work  performed  within  a  given  time  period.” 

3.  Budgeted  Cost  for  Work  Performed  (BCWP)  is  The  sum  of  the 
budgets  for  completed  work  packages  and  completed  portions  of  open 
work  packages,  plus  the  applicable  portion  of  the  budgets  for  level  of 
effort  and  apportioned  effort.” 

4.  The  Performance  Measurement  Baseline  (PMB)  is  The  time  phased 
budget  plan  against  which  contract  performance  is  measured.  It  is  formed 
by  the  budgets  assigned  to  scheduled  cost  accounts  and  the  applicable 
indirect  budgets.  For  future  effort,  not  planned  to  the  cost  account  level, 
the  performance  measurement  baseline  also  includes  budgets  assigned 
to  higher  level  contract  work  breakdown  structure  elements  and 
undistributed  budgets.  It  equals  the  total  allocated  budget  less 
management  reserve.” 

5.  Management  Reserve  or  Management  Reserve  Budget  is  “An  amount 
of  the  total  allocated  budget  withheld  for  management  control  purposes, 
rather  than  designated  for  the  accompiishmertt  of  a  specific  task  or  set  of 
tasks.  It  is  not  a  part  of  the  performance  measurement  baseline.” 

6.  Total  Allocated  Budget  is  The  sum  of  all  budgets  allocated  to  the 
contract.  Total  allocated  budget  consists  of  the  performance 
measurement  baseline  and  all  management  reserve.  The  total  allocated 
budget  will  reconcile  directly  to  the  contract  budget  base.  Any  differences 
will  be  documented  as  to  quantity  and  cause.” 

7.  Contract  Budget  Base  is  “The  negotiated  contract  cost  plus  the 
estimated  cost  of  authorized  unpriced  work." 
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8.  Budget  At  Completion  (BAG)  is  “The  sum  of  all  budgets  allocated  to 
the  contract.  It  consists  of  the  performance  measurement  baseline." 

9.  Estimate  At  Compietion  (EAC)  is  “Actual  direct  costs,  plus  indirect 
costs  allocable  to  the  contract,  plus  estimate  of  costs  (direct  and  indirect) 
for  authorized  work  remaining.” 
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Appendix  B:  Earned  Value  Measurement  Techniques 

The  50/50  Technique.  This  appreach  allows  the  contractor  to  claim  50 
percent  of  the  BCWP  when  the  work  package  is  open,  and  the  other  50  percent 
when  the  work  package  is  complete.  Some  contractors  modify  this  approach  by 
using  ratios  such  as  60/40  or  25/75  (7:122).  The  rationale  behind  the  original 
50/50  approach  is  that  on  average,  all  open  work  packages  are  roughly  50 
percent  complete,  making  the  result  of  this  approach  accurate.  This  is  also  one 
of  the  most  objective  estimation  techniques,  since  BCWP  is  determined  by 
whether  a  work  package  is  unopened,  opened,  or  completed. 

The  0/100  Technique.  This  approach  is  similar  to  the  50/50  technique  in 
the  calculation  of  BCWP.  In  this  case,  a  work  package  gets  all  of  the  BWCP 
when  it  is  complete.  This  method  would  tend  to  show  the  contractor  behind 
schedule,  and  is  best  for  work  packages  that  are  completed  within  one 
accounting  month.  (7:122) 

Milestone  Method.  This  method  works  best  for  work  packages  which  last 
more  than  a  few  months.  Objective  milestones  are  established  throughout  the 
performance  period,  and  the  contractor  receives  a  certain  percentage  of  the  total 
BWCP  when  those  milestones  are  met  (7:122). 

Percent  Complete.  This  approach  requires  a  monthly  estimate  of  open 
work  package  progress.  This  estimate  may  be  objective  or  subjective.  Objective 
percent  complete  is  determined  by  some  set  of  firm  guidelines.  As  an  example, 
one  may  claim  credit  for  half  the  construction  of  a  50  foot  fence  when  25  feet  are 
complete.  In  government  contracting,  where  there  are  often  very  complicated 
tasks  performed,  the  development  of  objective  measurements  is  often  not 
practical.  Therefore,  DOD  contractors  will  often  use  a  subjective  estimate  of 
percent  complete. 

While  the  subjective  percent  complete  technique  is  the  least  objective 
technique,  it  has  become  widely  used  in  industry.  The  hope  of  the  customer  is 
that  the  contractor  will  be  objective  in  his  estimates  and  have  professional 
integrity.  However,  due  to  the  possible  inaccuracy  of  this  approach,  it  is  best 
applied  to  work  packages  that  are  short  in  length.  (7:1 23) 

Equivalent  and/or  Completed  Units.  This  and  the  next  approach,  Earned 
Standards,  are  best  applied  to  manufacturing  efforts.  This  first  method  gives  a 
value  on  each  unit,  or  fractional  unit  produced.  For  example,  if  a  contractor 
builds  ten  tables  that  are  half  done,  five  equivalent  units  of  table  may  be  claimed 
and  half  of  the  BWCP  will  be  earned.  (7:123) 
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Earned  Standards.  This  technique  requires  "the  prior  establishment  of 
standards  for  the  performance  of  all  the  tasks  to  be  required”  (7:124).  Standards 
derived  from  historical  cost  data,  and  time  and  motion  studies,  must  be  available 
to  implement  this  measurement  method.  Earned  standards  measurement  is 
most  appropriate  for  repetitive  manufacturing  situations  (7:124). 
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